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I.  INTRODUCTION 

When  one  is  taking  measurements  with  the  same  instrument  of  succes- 
sive occurrences  of  the  same  phenomena,  one  can  then  compute  the  vari- 
ance and  mean  of  the  sample.  If  the  measuring  instrument  has  some  sort 
of  measurement  error,  this  error  becomes  part  and  parcel  of  the  sample's 
statistics  unless  one  can  somehow  "strip  out"  this  error  from  the  real 
variability  (product  variability)  and  real  mean  of  the  phenomena  being 
measured.  This  task  is  complicated  further  if  the  phenomena  are  non- 
replicatable  as  is  the  case  with  destructive  testing. 

When,  however,  two  or  more  instruments  independently  and  simultan- 
eously measure  successive  occurrences,  it  is  possible  to  estimate  both 
the  true  product  variability  and  the  precision  (or  imprecision)  of  meas- 
urement of  each  instrument  used.  These  methods  have  been  published  by 

Grubbs . ^ 

These  methods  were  later  expanded  to  the  case  where  one  wishes  to 
determine  whether  a given  measuring  instrument  is  operating  as  consis- 

2 

tently  as  two  other  known  instruments.  The  nomenclature  in  this  re- 
port is  consistent  with  that  found  in  the  referenced  reports. 

A minimum  of  two  instruments  simultaneously  measuring  is  needed. 

If  three  or  more  instruments  are  used,  estimates  of  precision  of  meas- 
urement can  be  made  which  are  totally  free  of  the  variability  of  the 
phenomena  observed.  This  results  in  more  direct  comparision  of  the 
errors  of  measurement  of  each  instrument.  The  procedures  discussed  here 
involve: 

(1)  The  estimation  of  the  precision  of  measurement, 

(2)  Estimation  of  product  variability  and, 

(3)  Significance  tests  for  comparing  the  precisions  of  measurement 
of  N instruments  (2  $ N $ 10)  and  whether  a given  instrument  has  a bias 
which  is  so  large  compared  to  that  of  the  other  measuring  devices  that 
the  instrument  needs  calibration. 


1 Grubbs,  F.E.,  "On  Estimating  Precision  of  Measuring  Instruments  and  Pro- 
duct Variability,"  Journal  of  the  American  Statistical  Association,  Vol. 
47,,  pp  243-264,'  June  1948. 

O ... 

Grubbs,  F.E.  and  O'Bryon,  J.F.,  "The  Statistical  Comparison  of  Measur- 
ing Instrumentation,"  BRL  ARDC  Technical  Report  No.  11  (AD  732  428), 

July  1971. 


II.  STATISTICAL  OVERVIEW 
A.  Statistical  Assumptions 

The  following  statistical  overview  is  reproduced  from  References  1 
and  2 to  enable  the  computer  program  user  the  opportunity  to  understand 
the  functioning  of  the  program.  References  are  made  in  the  examples  to 
equations  discussed  here.  For  a comprehensive  understanding  of  the 
statistical  basis  for  the  program,  the  reader  is  encouraged  to  study 
the  referenced  publications. 

We  shall  first  define  some  basic  terminology.  Below  we  see  tabu- 
lated pairs  of  data  points  measured  simultaneously  by  two  instruments, 
Ij  and  I?, 


Measurements  by  I. 


Measurements  by  I. 


X1  + ell 


X1  + el2 


x2  + en 


y + e 
2 22 


x.  + e. , 
i ll 


Xi  + Ci2 


x + e , 
n nl 


x + e t 
n n2 


where  x.  = the  true  value  of  the  characteristic  considered,  where  i rep- 
resents the  ith  item  in  the  sample, 

e. . = the  error  in  measurement  of  the  ith  item  by  the  jth  instru- 
1J  ment,  K . 

It  is  assumed  that  both  the  characteristic  being  measured  and  the 
errors  of  measurement  are  normally  distributed  and  that  the  absolute 
values  of  the  characteristic  measured  and  the  errors  of  measurement  are 
statistically  independent,  i.e., 


E(XiP  . e..^)  = E(xtP)  . Efe^), 
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In  addition,  it  is  assumed  that  the  errors  of  measurement  of  the 
different  instruments  are  statistically  independent  of  each  other,  i.e., 

E<eijP  • V>  ■ E‘eijP)  • Et.tkq>  > * k • 


It  is  also  assumed  that  the  errors  of  measurement  are  significantly 
smaller  than  the  real  value  of  the  characteristic  being  measured. 

B.  General  Case:  Estimation  of  Precision  of  Measurement  and  Product 

Variability : 

The  basic  way  in  which  the  computer  program  is  used  is  to  estimate 
the  precision  of  measurement  and  product  variability  given  observations 
from  2 to  10  instruments  simultaneously. 

Since  the  statistical  treatment  is  somewhat  different  for  the  two 
instrument  case,  this  statistical  summary  is  presented  in  two  parts. 

(1)  Two  Instrument  Case.  For  this  case,  there  are  two  methods  for 
estimating  precision  of  measurement. 

We  take  as  an  estimate  of  the  variance  in  errors  of  measurement  of 
instrument  I , 


est.  (o  , - S . _ 

el  x+el  x+el,x+e2 


(1) 


where 


x+el , x+e2 


i n 

= y 

n- 1 . L 


i = l 


(Xj  + e.j)  - (x  + e 


1 )|  j(xi  + ei 


-2) 


- (x  + e-) 


nCnViy  j " (xj  + eu)  (x2i  + e.2J 


f j (xA  - e )]  [( J (xi  + ei2)] 

i=l  i=l 


and  as  an  estimate  of  the  variance  in  errors  of  measurement  of  instru- 
ment  I we  compute 


est . 


^ x+e2  Sx+el,x+e2 


(2) 
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where  the  estimates  of  the  covariances 

E(Sx,el)  = E{Sx,e2)  = E(Sel,e2}  = 0 under  the  assumPtions- 

It  is  remarked  in  passing  that  a technique  of  working  with  differ- 
ences in  corresponding  measurements  of  the  two  instruments  is  of  some  in- 
terest. That  is,  if  the  measurements  of  are  subtracted  from  corres- 
ponding measurements  of  I , the  result  gives 

ell  " e12 
C21  - e22 


e , - e _ 
nl  n2 

thus  eliminating  the  true  value  of  x^  . , the  characteristic  measured. 

- 2 

Using  the  above  differences,  an  estimate  of  is  given  by 

est.  (a  ,2)  = i {S2  , - S2  . + S2  , . (3) 

el  2 1 x+el  x+e2  el-e2J 

It  is  easy  to  see  that  estimate  (3)  is 


10 


r 


s , + s -s  -s 

el  x,el  x,e 2 el,e2 

and  hence  precisely  that  given  by  (1). 

In  order  to  estimate  the  amount  of  variability  in  the  product  being 
measured,  we  may  first  add  corresponding  measurements  of  the  two  instru- 
ments, obtaining 


X1  + ell  + X1  + e12 


x.  + e.,  + x.  + e . _ 
1 i 1 i i2 


n nl  n n^ 
The  variance  of  the  above  sums  of  readings  is 


a ? -) 

4S“  + S , + S% 
x el  e2 


4S  + 4S  _ + 2S  , 0 . 

x,el  x,e2  el,e2 


An  estimate  of  the  variability  in  the  product,  o“,  is  then  given  by 


est.  (o2)  = 7 {S2 

x 4 x+el+ 


x+e2  ^el-e2-r  • 


(4) 


2 

The  estimate  of  a is  given 
I j and  I ^ 


simply  by  the  covariance  of  the  readings  of 


est . 


S 


x+el , x+e2 


(4a) 


(Equation  4 and  4a  are  identical). 

Estimate  (4)  or  (4a)  is  unbiased  and  can  be  shown  to  be  the  maximum 
likelihood  estimate. 

2 2 

The  variance  of  the  estimate,  est.  (a  ) or  est.  (o^^).  depends  on 

2 2 
(A)  a*,  the  variance  in  the  characteristic  measured,  (B)  , the  variance 
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2 

or  the  errors  of  measurement  of  Instrument  I,,  (C)  o“  , the  variance  of 

1 e2 

errors  of  measurement  of  Instrument  I 0 and  (U)  n,  the  number  of  obser- 
vations or  the  sample  size.  Therefore,  in  order  to  obtain  a precise  es- 
2 

timate  of  a , when  using  only  two  instruments,  the  variation  in  the 
el 

characteristic  measured,  i.e.,  o“ , should  be  held  to  a reasonable  minimum 

x 

or  the  sample  size,  n,  should  be  sufficiently  large. 


Sometimes  the  estimated  variance  in  errors  of  measurement  may  turn 
out  to  be  negative.  This  may  be  due  to  small  sample  size,  bad  observa- 
tions, errors  in  recording,  outlying  observations,  etc.  Since  it  is 
physically  impossible  for  a true  variance  in  errors  of  measurement  to  be 
negative,  one  suggestion  is  to  add  the  absolute  value  of  the  negative 
variance  to  the  estimated  variance  in  error  for  the  other  instrument  for 
an  overall  estimate  of  instrument  error.  The  computer  program  prints 
negative  as  well  as  positive  variances  and  stores  these  values  to  compute 
pooled  estimates  of  variance  in  precision  error  for  each  instrument  if 
multiple  cases  are  input  back-to-back  using  the  same  instruments.  How- 
ever, to  avoid  negative  square  roots,  the  program  lists  °e  = 0 when  tak- 
ing the  square  root  of  a negative  variance.  See  References  1,  2 and  4 
for  further  information  on  the  treatment  of  negative  estimates  of  vari- 
ance . 

If  the  variation  in  the  characteristic  measured  is  zero  (or  if  we 

2 

measure  the  same  item  over  and  over  again)  i.e.,  if  a = 0,  then  one  could 
compute 


1 

n- 1 


I 


i = 1 


(e 


il 


(4b) 


directly , 


and  this  would  give  an 


est imate 


of  a 


2 

el 


with  variance  equal 


to 


2 

n-1 


(4c) 


Apparently,  in  employing  two  instruments,  there  are  only  two  compu- 
tational procedures  of  interest  for  separating  the  variability  in  the 
product  from  the  variance  in  the  errors  of  measurement  and  both  methods 
give  the  same  estimate.  In  using  either  method,  however,  it  is  possible 
2 2 2 

to  estimate  °ej>  °e2  ar>d  °x  and  thus  determine  from  the  relative  order 
_ . 

Thomson,  W.A.,  Jr.,  "The  Problem  of  Negative  Estimates  of  Variance  Com- 
ponents," Annals  of  Mathematical  Statistics;  Vol  33,  pp  273-288,  1.9 64. 
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of  magnitude  of  these  quantities  whether  the  instruments  are  sufficiently 
precise  to  use  in  taking  the  required  measurements. 

(2)  Three  or  More  Instrument  Case  (3  < N <_  10).*  Since  the  nomen- 
clature has  been  established  in  the  two  instruments,  this  description  of 
the  N _>  3 case  will  be  limited  to  the  basic  equations  solved  by  the  com- 
puter program.  As  was  stated  earlier,  using  three  or  more  instruments 
permits  precision  of  measurement  estimates  totally  free  of  the  product 
variability  itself. 


The  following  equation  is  used  to  estimate  precision  of  measure- 
ment of  Instrument  I (2  < i < 5C  data  points) 


2 2 

est . (o  ) = S 

el  x+el 


— 1 } S , 1 

N-l  x+el,x+erJ 

r=2 

, k=N 

+ £ j y s 1 

(N-l ) (N-2)  2^i<k  x+cJ’x+ek' 


(5) 


Estimates  of  all  other  instruments  can  be  made  by  obvious  rotations  of 
the  subscripts.  The  real  product  variability  (stripped  of  measurement 
error)  is  estimated 


est . 


2 

N (N-l)' 


s=N 


lfr^s 


S 

x+cr ,x+es 


(6) 


C.  Comparative  Case:  Determination  of  Relative  Performance  of  Two  "Known" 

Instruments  Versus  One  "Unknown"  Instrument . 

The  application  considered  here  is  that  of  comparing  an  instrument 
of  unknown  performance  characteristics  with  those  of  two  known  or  proven 
test  instruments  to  determine  if  their  performance  is  statistically  dif- 
ferent . 


If  we  measure  a series  of  occurrences  with  three  instruments  simul- 
taneously, it  can  be  seen  that  the  difference  in  readings  of  any  two  of  the 
instruments  consists  of  only  the  differences  in  (1)  errors  of  measurement 
and  (2)  constant  biases  for  the  two  instruments,  and  is  hence  free  of  the 
level  of  parameter  measured.  The  model  used  here  is  based  on  this  concept. 
From  the  variation  or  variance  of  the  differences  in  errors  of  measurement 
from  point  to  point  of  pairs  of  the  three  instruments  making  simultaneous 
_ - - - 

N is  limited  to  10  and  n to  SO  only  because  of  the  dimensions  of  the  pr  - 
gram's  storage.  The  equations , of  course,  are  valid  for  any  N f 3,  • > 
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measurements,  one  can  strip  out  and  hence  estimate  the  variance  in 
errors  of  measurement  of  each  of  the  three  instruments.  These  figures 
therefore  are  basic  to  determining  whether  the  precision  of  measurement 
is  suitably  good  as  compared  with  an  estimate  of  the  variance  of  true 
readings  in  order  to  judge  precision  of  measurement  properly.  This, 
along  with  a comparison  of  levels  of  readings  measured  by  the  instrument, 
will  determine  accuracy  or  the  need  for  calibration.  Moreover,  statis- 
tical tests  of  significance  for  precision  and  accuracy  of  measurement  are 
rather  easily  developed  and  applied,  as  will  be  seen. 

The  statistical  or  mathematical  model  used  for  the  observed  measure- 
ments of  the  three  instruments  is  indicated  by  the  quantities. 


8.  + x.  + e. , 
1 1 ll 


32  + Xi  + Ci2  = Si 


33  + Xi  + Ci 3 = Ti 


where  8^,  82  and  8^  are  respectively  the  unknown  instrumental  biases, 

e.,,  e.,.  and  e._  the  random  errors  of  measurement  of  the  three  instru- 
ll  i2  i3 

ments,  and  x^  the  true  unknown  reading  of  the  ith  data  point  or  charac- 
teristic measured  in  a series  of  measurements  on  n data  points. 


In  order  to  estimate  the  standard  deviation  of  the  true  readings 
measured,  one  may  use  the  following  formula  which  gives  an  unbiased  es- 
timate of  the  true  reading  variance 


est.  (o  ) = (S  . - + S - 7 + S . _)/3 

x x+el,x+e2  x+e2,x+e3  x+el,x+e3 


(7) 


where  the  S's  are  sample  covariances  of  the  three  instruments,  taken  two 
at  the  time,  each  based  on  n-1  degrees  of  freedom. 

For  the  three  instrument  case,  variance  in  precision  error  for  first 
instrument  is 


est.  (o'"  ) = S + S _ _ - S , -,-S  , _.  C 7a) 

el  x+el  x+e2,x+e3  x+el,x+e2  x+el,x+e3 


Again  estimates  for  the  other  instruments  are  made  by  simply  "rotating  the 
subscripts"  of  the  above  equation.  We  next  wish  to  determine  whether  the 
two  "known"  instruments  are  performing  consistently  well. 

Two  Student  t- tests  are  outlined  which  help  to  determine  whether  the 
two  standard  instruments  chosen  are  performing  consistently  well  to  en- 
able them  to  make  a reliable  judgment  concerning  the  performance  of  a 
third  unknown  instrument  submitted  for  study. 
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The  first  test  examines  the  two  standards  for  relative  precision. 
Definition  of  terms  used  in  the  following  equations  are  described  in 
Reference  2.  These  instruments  should  be  roughly  equivalent  in  precision 
to  be  used  as  a basis  of  comparison.  If  this  t-test  shows  no  significant 
difference  in  precision  of  measurement,  the  levels  of  precision  should 
still  be  examined  to  assure  that  the  error  of  precision  is  much  less  than 
the  actual  parameter  variation  itself.  The  test  takes  the  following  form 


t = rtycl/rTT 

J T7 
1-r  Cyz) 


(8) 


We  should  now  define  some  more  terms  in  the  order  presented, 

y = the  difference  between  the  sum  of  mean  readings  of  the  two 

standard  instruments  and  the  sum  of  the  ith  readings  of 
the  same  instruments. 


z = Vj  = the  difference  in  readings  between  the  two  standard  instru- 
ments, 


r(yz)  = the  sample  correlation  coefficient  of  y and  z, 

S(z)  = the  standard  deviation  of  the  z sample, 

u = the  difference  between  the  test  instrument  reading  and  the 

average  of  the  readings  of  the  two  standards. 

The  second  test  (Equation  9)  looks  at  the  same  two  instruments  for 
average  level  of  readings.  If  the  t-test  detects  a significant  differ- 
ence, recalibration  should  be  performed  to  bring  the  instruments  into 
better  agreement 


t0  = z/n/S(z) 


(9) 


We  might  say  here  that  occasionally  this  t-test  (Equation  9)  for 
average  levels  detects  a significant  difference  even  for  very  small  dif- 
ferences in  average  readings.  This  is  caused  usually  by  the  influence 
of  the  very  small  standard  deviation  in  the  difference  in  the  readings 
of  the  two  known  instruments  over  the  series  of  occurrences. 

Once  both  t-tests  have  been  passed,  it  can  be  established  that  the 
standard  instruments  are  performing  consistently  with  each  othei . 

Since  one  many  times  has  no  way  of  knowing  the  real  data  values, 
(i.e.,  destructive  testing)  the  assumption  is  made  that  the  best  estimate 
of  the  real  value  is  midway  between  the  two  standard  measurements.  Hence, 
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the  average  readings  from  these  two  standard  instruments  are  computed  and 
compared  with  the  observations  gathered  from  the  unknown  instrument  sub- 
mitted for  study. 


Tests  for  precision  and  bias  (similar  to  those  just  outlined  to 
determine  a valid  test)  are  performed,  this  time  comparing  the  average  of 
the  standard  readings  with  readings  from  the  third  instrument. 


t 

o 


(n-2) 


[s2 Cups2 Cv:)  - .75]4TT 
[3  (l-r“  (u^))  (S“  (u^)S^  (v1) ) ] 


1/2 


(10) 


If  the  t-test  for  precision  detects  a significant  difference  (Equa- 
tion 10)  one  should  determine  why  the  test  showed  significance.  Mere 
failure  of  the  test  does  not  constitute  an  inferior  performance  by  the 
third  instrument.  If  the  estimate  of  error  in  precision  of  the  unknown 
instrument  (T)  is  actually  much  less  than  the  average  of  the  two  standards, 
the  third  instrument  has  an  acceptable  precision,  even  though  the  t-test 
showed  significance.  In  fact  its  precision  appears  to  be  superior  to  the 
standards . 


Finally,  the  levels  of  measurement  of  the  unknown  instrument  are  com- 
pared with  the  average  readings  from  the  standard  instruments. 


tQ  (n-1)  = Uj^r/SOij)  (11) 

This  statistical  procedure  is  flexible  enough  to  use  with  any  number 
of  instruments  and  not  limited  to  the  three  described  above.  Applying 
these  tests  enables  those  interested  to  generate  a "data  bank"  of  typical 
instrument  performance. 

It  is  possible,  after  groups  of  data  points  are  gathered,  to  combine 
the  estimates  of  variance  in  precision  error  to  obtain  a better  estimate 
of  the  overall  performance  of  each  instrument.  This  is  done  automatically 
by  the  program  when  cases  are  stacked  successively  with  measurements  from 
the  same  instruments  recorded  in  the  same  data  fields. 

D.  Data  Handling  of  Empty  Cells  and  Outliers 

One  very  important  element  of  the  analyses  is  the  handling  of  missing 
data  and  extreme  observations  or  outliers. 

1 . Missing  Data  and  Empty  Cells. 

If  data  in  one  or  more  cells  (a  cell  being  simply  one  measurement  by 
one  instrument)  is  missing,  it  is  necessary  that  something  be  done  to  as- 
sure that  the  data  point  is  not  considered  as  a reading  of  zero  (unless 
zero  is  a valid  reading). 
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A simple  means  would  be  to  omit  all  other  measurements  of  that  same 
data  point  by  the  other  instruments  (one  row  of  data).  Alternately,  one 
could  eliminate  all  other  measurements  by  that  instrument  which  has  an 
empty  cell  (one  column  of  data).  Provision  is  made  in  the  program  to  do 
either  or  both  of  these  things  by  means  of  control  cards  2 and  3 of  each 
case.  One  should  keep  in  mind  the  deletion  of  an  entire  row  or  column 
reduces  the  degrees  of  freedom  by  the  proportionate  amount. 

An  alternate  approach  to  avoid  having  to  eliminate  an  entire  row  or 
column  of  data  due  to  a missing  data  point  would  be  to  estimate  the  sice 
of  the  missing  value  based  on  standard  ANOVA  procedures  and  substitute 
the  estimated  value  for  the  missing  data  point.  This  would  also  reduce 
by  one  the  degrees  of  freedom  for  the  significance  tests  performed  by 
the  program.  Since  the  computer  program  has  no  way  of  differentiating 
estimated  values  from  actual  readings,  the  degrees  of  freedom  must  be 
manually  reduced  on  the  computer  output  when  observations  are  estimated. 

2 . Uxtreme  Values  and  Out  1 i ers . 

Since  the  precision  estimates  performed  by  the  program  are  sensitive 
to  the  presence  of  outliers  or  extreme  values,  it  is  imperative  that  the 
program  user  deal  with  their  presence. 

Built  into  the  program  is  an  outlier  test  (a  modified  F-test)-3  which 
flags  any  value  which  is  statistically  determined  to  be  an  outlier  at  the 
.95  level  of  confidence. 

Tests  are  performed  on  columns  (all  observations  by  an  instrument) 
and  on  rows  (observations  of  one  data  point  by  all  instruments).  If  an 
outlier  is  detected  it  is  flagged  by  the  placement  of  an  "I"  if  a column 
outlier  and  by  a "P"  if  a row  (or  point)  outlier.  An  "IP"  may  appear  if 
detected  by  both  tests.  Naturally  any  missing  data  (empty  cell)  not 
omitted  by  the  control  card  options  (cards  2 or  3)  will  be  flagged  as 
outliers.  In  the  comparative  case  (two  known  instruments  versus  one  un- 
known), outliers  are  flagged  by  means  of  an  asterisk 

It  is  important  to  note  that  the  detection  of  an  outlier  does  not 
dictate  its  automatic  removal  from  consideration  as  a data  point.  In 
fact,  it  may  be  a very  valuable  data  point.  It  is  up  to  the  program 
user  to  decide  how  to  deal  with  the  extreme  values.  If  it  is  due  to  a 
typographical  error  or  human  recording  error,  it  may  be  amendable.  If, 
on  the  other  hand,  there  is  no  rational  reason  to  doubt  the  observer's 
authority,  nor  to  expect  the  instrumentation  to  behave  any  differently 
on  future  tests,  it  might  be  wise  to  include  the  outlying  observation  as 
an  acceptable  data  point  in  computing  typical  instrument  performance. 

It  is  suggested  that  on  the  first  computer  run,  omit  only  data  from 
empty  cells  and  let  the  program  flag  all  outliers.  Then  the  decision  can 
be  made  as  to  how  to  deal  with  these  values  on  subsequent  runs. 

~2  . . 

Grubbs , F.E.,  "Procedures  for  Detecting  Outlying  Observations  in  Samples, 

BKL  Report  No.  1713,  AD  781  499,  April.  1974.' 


The  only  way  that  an  entered  data  point  may  be  ignored  from  compu- 
tation is  by  means  of  control  cards  2 and  3 described  in  the  next  sec- 
tion. Flagging  a data  point  by  the  computer  does  not  constitute  the 
disqualification  of  that  point,  nor  does  the  presence  of  a flag  (I,  P 
or  *)  remove  the  data  point  from  the  computational  procedure.  Removal 
of  a data  point  can  only  be  accomplished  by  means  of  the  control  cards. 


III.  EXAMPLES 

A.  Example  1:  General  Case 

This  case  consists  of  12  projectile  firings  with  muzzle  velocity 
measured  by  9 different  instruments.  Measurements  of  the  sixth  occur- 
rence have  been  deleted  since  the  FBI01  (B)  failed  to  obtain  a reading 
(Figure  1).  If  the  empty  cell  were  allowed  to  be  entered,  the  precision 
estimates  would  be  significantly  affected.  This  is  illustrated  through 
data  gathered  by  the  NM87A  instrument  (H) . It  failed  to  measure  the 
fourth,  ninth  and  tenth  occurrences.  These  empty  cells  were  entered  as 
data  points  and  as  a result,  the  inflated  estimate  of  variance  in  pre- 

2 4 

cision  (est.  a ) = 11.6  x 10  . 

6rl 

Also  notice  that  the  second  observation  made  by  the  COMP  (C)  in- 
strument has  been  detected  as  an  outlier  at  a confidence  level  of  95% 
when  compared  with  observations  of  the  same  occurrences  by  the  other  in- 
struments. The  same  is  true  of  six  measurements  by  the  TERMA  1 (F)  in- 
strument . 

Due  to  the  presence  of  the  extreme  values  and  empty  cells,  the  pre- 
cision estimates  are  highly  different.  Hence  we  use  the  case  as  a com- 
puter run  to  detect  outlying  observations  and  missing  data.  Shaded  data 
indicates  information  deleted  by  control  cards. 

B.  Example  2:  General  Case:  Data  Deleted 

Based  on  the  results  of  Case  1,  we  have  decided  to  delete  all  ob- 
servations from  the  TERMA  1 (F)  and  NM87A  (H)  due  to  their  erratic  be- 
havior. Data  point  6 is  also  deleted  from  consideration  from  all  instru- 
ments due  to  a missing  observation  from  the  FBI01  (B)  (Figure  2).  Hence 
we  are  basing  our  results  on  measurements  from  7 instruments  over  11 
points  for  a total  of  77  observations.  Two  outliers  are  included  since 
it  was  felt  that  these  were  real  data  points  indicative  of  the  perform- 
ance of  the  instruments  involved  (C  and  G) . 

The  analysis  indicates  that  the  COUNTER  (A),  FOTOCEL  (E) , FBI02  (D) 

2 

and  NM87B  (I)  have  the  best  precision  of  measurement  (lowest  est.  a ^) . 

In  fact  two  of  the  estimates  have  negative  values  (COUNTER  and  FOTOCEL). 
This  is  possible  since  the  sample  size  is  rather  small  and  we  are  com- 
puting the  best  estimate  of  precision,  that  estimate  having  a chance  of 
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being  above  or  below  the  real  value.  If  est.  (a  ) is  negative,  the 

computed  est.  (a  . ) is  set  at  zero  on  the  computer  printout.  However, 
el  2 

all  values  of  est.  C°e^)»  positive  and  negative  are  stored  in  the  pro- 
gram to  compute  a pooled  estimate  of  precision  of  measurement  if  cases 
involving  the  same  instruments  are  stacked. 


We  see  the  estimate  of  the  actual  parameter  variance  (stripped  of 

2 

measurement  error)  as  being  est.  a = 6.009.  Hence  the  variance  in  pre- 
cision of  measurement  is  orders  of  magnitude  less  than  the  parameter  vari- 
ance point-to-point.  In  other  words,  for  this  example,  the  variation 
point-to-point  is  due  largely  to  real  product  variability  and  not  to 
measurement  error. 


C.  Example  3:  Comparative  Case:  Two  "Known"  Instruments  Versus  a 

Third  "Unknown"  Instrument 


In  this  example,  we  are  attempting  to  decide  whether  some  "unknown" 
or  "untested”  instrument  COMP  (C)  performs  better  than  the  average  of 
two  "known"  or  tried-and-tested  instruments  (COUNTER,  (A)  and  FBI01  (B)) 
(Figures  3 and  4) . 

Since  we  want  to  limit  our  invest igation  to  only  these  three  instru- 
ments, all  other  data  has  been  deleted.  The  sixth  data  point  has  also 
been  removed  from  consideration  since  an  empty  cell  exists. 

Figure  4 lists  12  occurrences  as  measured  simultaneously  by  three 
instruments  (R,  S,  T) . The  R and  the  S represent  the  two  proven  stand- 
ards, whereas  the  T represents  an  unknown  instrument  submitted  for  test. 
Figure  4 gives  for  firings  in  each  combination  of  two  instruments  the 
individual  and  mean  readings  of  the  three  instruments,  the  individual 
differences  in  measurement  for  the  instruments  taken  two  at  a time  and 
the  computational  procedures  for  estimating  the  variance  in  errors  of 
measurement  for  each  instrument,  and  the  estimated  variance  of  the  true 
readings  measured  by  each  of  the  instruments. 

These  tests  reveal  that  their  variances  in  precision  error  are  not 
significantly  different  (.988  < t 1.833)  but  their  mean  values  are 

significantly  different  (4.052  > t (J5  1.812)  (730.87  versus  730.16). 

This  indicates  that  they  are  in  need  of  calibration. 

Finally  we  proceed  to  compare  the  average  performance  of  the  "known" 
standard  instruments  with  the  "unknown"  instrument.  The  second  set  of 
t-tests  are  performed  on  means  and  precisions.  In  the  example,  there 
were  no  significant  differences  (if  no  calibration  is  performed)  between 
the  combined  means  of  the  two  "known"  instruments  (730.5)  and  the  "unknown" 
instrument  mean  (730.2).  However,  the  statistics  show  that  the  "unknown" 

• , 2 

instrument  (COMP  (C))  was  significantly  less  precise  (higher  est.  a .) 

Cl 
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THE  MEANS  rr  THE  STANOARO  INSTRUMENTS  ARE  SIGNIFICANTLY  OlfEfRENT 
THE  TEST  INSTRUMENT  is  5 I &N I F I C ante  Y LESS  PRECISE  THAN  THE  STA'H'AROS 


than  the  standard  (7.66  versus  -.18  and  .52)  since  the  calculated  value 
t (8.23)  exceeds  the  t critical  at  .95  with  9 degrees  of  freedom  (1.833). 


IV.  DESCRIPTION  OF  DATA  INPUT 
A. 

Card  1 

Cols  1-2  - Number  of  data  points  for  each  instrument  (integer) 

(total  number  of  data  cards  to  be  read  in  with  a mini- 
mum of  2 and  maximum  of  50) . (The  total  number  of  data 
cards  to  be  read  includes  data  cards  excluded  by  control 
card  3) . 

Cols  3-4  - Number  of  instruments  making  simultaneous  measurements 

(integer)  with  a minimum  of  2 and  maximum  of  10.  The 
total  number  of  instruments  recording  data  includes  in- 
strumentation excluded  by  control  card  2. 

Card  2 

Cols  1-10  - Insert  the  digit  1 in  every  column  starting  in  column  1 
out  to  the  column  indicating  the  number  of  instruments 
used,  data  for  any  instrument  may  be  omitted  from  analy- 
ses by  placing  a zero  rather  than  1 in  the  column  cor- 
responding to  that  instrument.  Example:  111101  would 

mean  use  data  from  all  instruments  except  instrument  5 
(field  5). 

Card  3 

Cols  1-50  - Insert  the  digit  1 in  every  column  starting  in  column  1 
out  to  the  column  indicating  the  number  of  data  cards. 
Data  for  any  observation  may  be  omitted  from  analyses 
by  placing  a zero  rather  than  1 in  the  column  corres- 
ponding to  that  data  point.  Example:  1001111111  in 

columns  1-10  would  mean  to  use  all  data  cards  except 
those  of  observations  2 and  3 (data  cards  2 and  3) . 

Cards  4 and  5 


Cols  1-80  - Alphanumeric  titles. 
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Card  6 


Cols  1-8,  9-16,  17-24  ...  73-80  - Alphanumeric  headings  of  names  of 
instruments.  These  headings  must  be  identical  and  in 
identical  fields  from  case-to-case  if  several  cases  -re 
stacked  and  pooled  estimates  of  precision  error  are 
sired.  New  headings  on  this  card  will  cause  the  progiam 
to  pool  estimates  of  precision  on  most  recently  pro- 
cessed cards  with  other  headings. 

Card  7 Data  cards  (up  to  SO  data  cards,  one  card  per  data 

point,  a maximum  of  ten  simultaneous  measurements  by 
different  instruments  per  point,  (ten  fixed  point,  8 
digit  fields). 

Cols  1-8  - Observation  from  instrument  A of  first  occurrence, 

(fixed  point) 

Cols  9-16  - Observation  from  instrument  B of  first  occurrence, 

(fixed  point) 

Cols  17-24  - Observation  from  instrument  C of  first  occurrence, 

(fixed  point) 


Cols  73-80  - Observation  from  instrument  J of  first  occurrence, 

(fixed  point) 

Last  Card  Per  Case 

Option  1.  General  Case : estimation  of  variances  in  errors  of  pre- 

cision of  j instruments  (2  < j < 10) 

Col  1 - Insert  decimal  point  in  first  column. 

Option  2.  Comparative  Case:  Estimation  of  variances  in  precision 

of  measurement  and  also  computation  of  Student  t to 
determine  significance  of  differences  in  precision 
errors  and  bias  errors. 

I.  Parametric  mode  (to  look  at  all  combinations  of  three  instruments ) 

Cols  1-4  - Insert  '***.'  (three  asterisks  and  decimal  point) 

II.  Specific  mode  (to  look  at  specific  combinations) 

Cols  1-4  - Insert  'ABC.'  where  A,B,C  can  be  any  three  alphanumeric 

letters  referring  to  data  from  instruments  in  those  cor- 
responding to  those  letters  as  follows  (A-J)  : 
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Instrument 

A = data 

in 

cols 

1-8 

Instrument 

B = data 

in 

cols 

9-16 

Instrument 

C = data 

in 

cols 

17-24 

Instrument 

D = data 

in 

cols 

25-32 

Instrument 

. data 

in 

cols 

Instrument 

J = data 

in 

cols 

73-80 

The  first  two  letters  specified  are  considered  as  the  standards  (A, 
B)  and  the  third  letter  (C)  is  considered  the  nonstandard  test  instru- 
ment. The  letters  need  not  be  in  alphabetical  order,  an  asterisk  (*)  may 
be  used  in  place  of  any  one  or  two  letters  which  then  forces  a repeated 
analyses  with  data  for  every  other  instrument  to,  in  turn,  be  substituted 

for  that  asterisk  point.  Example:  BC*,  = BCA,  BCD,  BCE and  *C* , 

= ACB,  ACD,  BCA,  BCD.,,  DCA,  DC B ... 

Cases  may  be  stacked  with  a limit  of  50  cases  per  computer  run. 

A blank  card  must  be  placed  at  end  of  last  case.  This  also  causes 
the  program  to  compute  a summary  page  of  previous  cases  including  pooled 
estimates  of  precision  error.  All  blank  fields  within  the  data  card  are 
considered  as  zero  observations.  If  data  gaps  exist  for  one  or  more  in- 
struments, a zero  corresponding  to  that  instrument  or  occurrence  on  con- 
trols cards  2 or  3 causes  the  program  to  skip  over  incomplete  data.  A 
zero  in  a field  omits  all  data  for  that  instrument  (data  cards).  A zero 
in  a field  of  card  3 omits  that  observation  from  all  instruments.  Out- 
liers can  also  be  omitted.  The  program  checks  for  outliers  on  all  data 
which  are  not  already  removed  by  using  zeros  on  cards  2 and/or  3. 

B.  Output  Descriptions 

Headings 

Data  listing  (column  to  farthest  right  consists  of  row  mean  values) 

Mean  = mean  value  of  data  group  of  each  instrument 

Variance  = sample  variance  of  each  data  group 

Stnd  dev  - standard  deviation  of  data  group  of  each  instrument 

Prob  err  = sample  probable  error  of  each  data  group 

Grand  mean  = mean  of  all  observations  from  all  instruments 

Average  variance  = arithmetic  average  of  variances  from  each  instru- 
ment . 

Stnd  dev  = square  root  of  average  variance 
Prob  err  = stnd  dev  * .6745 
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Outlier  tests  are  performed  on  rows  and  columns  of  data,  with  de- 
cisions based  on  Modified  P-test  referenced  earlier  at  .95  level  of  con- 
fidence. Outliers  are  only  flagged.  They  are  not  deleted  by  the  program. 

Covariance  matrix  = sum  of  covariances  of  all  combinations  of  two 

instrumens . 

Cov  incl  inst  = sum  of  covariances  involving  the  instrument  in  that 
column.  All  other  instruments  considered. 

Cov  excld  inst  = sum  of  all  other  covariances  listed  in  covariance 
matrix/2 

Est  (sigma“j)  = unbiased  estimate  of  variance  in  errors  of  precision 
of  instrument  listed  in  that  field.  (Solution  to 
Equation  (2)  or  (5)  depending  on  number  of  instru- 
ments . ) 

Precision  ranking  = ranks  assigned  to  size  of  precision  ^rrors  with 

1 being  the  best  precision  (lowest  oj\  value) 

Total  covariance  = (sum  of  all  covariances  in  table)/2 

Parameter  variance  = unbiased  estimate  of  real  values  of  parameter 

being  observed.  (Solution  of  Equation  (4)  or 
(7)  depending  on  number  of  instruments.) 

Comparative  Case:  Two  standard  instruments  versus  one  nonstandard. 

R = data  from  first  standard  instrument 

S = data  from  second  standard  instrument 

T = data  from  nonstandard  instrument 

Y = difference  between  sum  of  mean  readings  of  R and  S and  sum  of 
ith  readings  of  R and  S 

Z = VI  = difference  in  readings  between  two  standard  instruments 
U1  = difference  between  instrument  T and  average  of  R and  S 
S(YZ)  = covariance  of  Y and  Z 
S (UV)  = covariance  of  U1  and  VI 

R(YZ)  = sample  correlation  coefficient  of  Y and  Z 
R(UV)  = sample  correlation  coefficient  of  U and  V 


T(STNDS  VAR)  ■=  computed  t statistic  from  Equation  (8)  to  determine 
if  the  standard  deviation  of  precision  errors  of  R 
and  S are  significantly  different 

T (STNDS  BIAS)  = computed  t statistic  from  Equation  (9)  to  determine 
if  bias  is  significant  between  the  two  standards  R 
and  S 

Statistics  to  determine  whether  standards  are  in.  close  enough  agree- 
ment in  precision  and  average  reading  (bias): 

T(TEST  VAR)  = computed  t statistic  from  Equation  (10)  to  determine 
if  the  nonstandard  test  instrument  is  operating  with 
a precision  of  measurement  equal  to  the  average  of  the 
two  standards. 

T(TEST  BIAS)  = computed  t statistic  from  Equation  (11)  to  determine 
if  the  nonstandard  test  instrument  is  biased  when 
compared  with  the  average  readings  of  the  two  stand- 
ards . 

Statistics  to  determine  whether  nonstandard  third  instrument  (T) 
has  a statistically  different  performance  in  precision  of  measurement 
and  average  reading  (bias)  from  the  two  known  standard  instruments  (R 
and  S ) : 

T.95  = Student  t value  to  be  used  as  a comparison  against  T(STNDS 
VAR)  and  (TEST  VAR)  to  determine  acceptance  of  null  hypo- 
thesis. T.95  is  to  be  used  as  a comparison  against  T(STNDS 
BIAS)  and  (TEST  BIAS), 

* = indicates  outlier  in  column  of  data  at  .95  confidence  level, 

R-S  = difference  between  observations  of  instruments  R and  S, 

S-T  = difference  between  observations  of  instruments  S and  T, 

T-R  = difference  between  observations  of  instruments  T and  R, 

Appendix  A is  a complete  program  listing  in  the  FORTRAN  IV  language. 

Appendix  B is  a listing  of  the  data  used  for  the  three  examples 
shown. 
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Appendix  A 
Program  Listing 

MCAAO  OBRYON  A271  39*.  t S T I M A I E PRECISION  OF  MEASUREMENT  FEB  77 


L I ST ( ST  AR  T ) MAIN 

DIMENSION  ASTR(SO)  l DIMENSION  N1SI50),  N2S(50)  MAIN 

DIMENSION  PCTVEU50I,  PCTVCY(50)  t DIMENSION  PEP(IO)  MAIN 

DIMENSION  P E S ( 10*  SOI  3 DIMENSION  LINSTRUO!  MAIN 

DIMENSION  Also,  10),  SUM A 110),  COVIIO.IO),  VARI10),  C0VAI10),  MAIN 

1 COVNA(IO),  PRBtlOI,  PEI  10),  INSTRUO),  SDEVIIO),  INCISTUO),  MAIN 

2 INCRN0I50I,  ISTNAMtlOl,  ISTDEL(IO),  SRPI80),  TITIEI16I,  MAIN 

3 IRNDELISOI  3 DIMENSION  ME AN A ( 10)  $ DIMENSION  1DELI2)  MAIN 

DIMENSION  T95IA9I  MAIN 

DIMENSION  Y I SO  ) , Z ( 50  J , UI50I,  $MT(50I,  TMRI50),  RSTMI50I  MAIN 

DIMENSION  AP E I 10),  IPE(IO)  3 DIMENSION  PEVI10.50)  MAIN 

DIMENSION  TTVI  10,  10),  TTLUO.IO),  TVLI10.10),  TLLI10.10)  MAIN 

DIMENSION  V150),  RTL150.10),  CTH50.10)  3 DIMENSION  MEANSI50)  MAIN 

DIMENSION  JPEI10)  3 DIMENSION  AC  R OS  S ( 50 ) MAIN 

DIMENSION  YF<50),  ZF(50),  UF(SO),  SMTFI50I,  TMRFI50)  MAIN 

DIMENSION  RSV(  10,  101 , RSLIIO.IO),  SVLU0.10),  SLLU0.10)  MAIN 

DIMENSION  POOLV(IO),  NPLD(IO)  3 DIMENSION  POOLSIIOI  MAIN 

INTEGER  R.S.T  3 EQUIVALENCE  <S,KS)  3 INTEGER  GRP  MAIN 

REAL  MEANA,  MEAN,  1ST  3 REAL  MEANS  MAIN 

DATA  ISTNAM  /'A',  '8',  'C',  'O',  'E',  *F',  'G',  'H',  'I',  ’J'/  MAIN 

DATA  I DEL  /'DELETED',  "/  3 DATA  KY,  Kl,  KU,  KSMT , KT  MR  75*1/  MAIN 


DATA  T95  / 6 . 3 1 A , 2.920,  2.353,  2.132,  2.015,  1.9A3,  1.895,  1.860,  MAIN 
11.833,  1.812,  1.796,  1.782,  1.771,  1.761,  1.753,  1.7A6,  1.7A0,  1.7MAIN 
23 A , 1.729,  1.725,  1.721,  1.717,  1.71A,  1.711,  1.708,  1.706,  1.703, MAIN 
3 1.701,  1.699,  1.697,  1.695,  1.693,  1.692,  1.690,  1.689,  1.688,  l.MAIN 
A687,  1.686,  1.685,  I.68A,  2.683,  2.682,  ! . 682  , 1.681  , 1.680,  1 .679MA1N 


5,  1.679,  1.678,  1.678/  3 NSAV-1  MAIN 

WRITE  I 6 , 1 A 8 ) 3 ABCD='MEAN*  MAIN 

1 READ  (5,96)  MAXRND.MAX I ST  3 IF  ( MAXRND. EQ. 0)  GO  TO  A MAIN 

READ  (5,91)  ( INCISTl  I),l=l,MAXIST)  MAIN 

READ  (5,91)  ( INCRNDI  I), 1=1, MAXRND)  3 READ  (5,93)  ( T I TLE ( I I , I = 1 , 16  I M A IN 
READ  { 5,  9 A ) l INSTRI  I I, 1 = 1, MAXI  ST)  MAIN 

DO  2 1=1, MAXRND  MAIN 

2 READ  (5,95)  ( A ( I , J ) , J = l , M AX  I ST ) 3 IF  (NSAV.EQ.50)  GO  TO  A MAIN 

MX  = M AX  1ST  3 IF  ( L MX  I ST .L T . MX ) MX  = LMXIST  MAIN 

DO  3 1 = 1,  MX  3 IF  ( INSTRI  I 1 .NE .L INSTR ( I ) ) GO  10  A MAIN 

3 CONTINUE  3 GO  TO  18  MAIN 

A IF  (NSAV.EQ.l)  GO  TO  17  3 WRITE  (6.1A9I  ( t I N STR 1 1 I , I = 1 , LMX I ST  I MAIN 

PCTVCYIJ 1=0.0  3 T OPC  T V = 0 . 0 3 N3S  = 0 3 NAS=0  MAIN 

DO  6 J= 1 , NSAV  3 PCTVEL I J ) =0.0  MAIN 

CO  5 1 = 1, LMX  1ST  3 PC  T VEl  ( J ) =P  E S ( I . J ) ■*  P C T Vi  I.  ( J)  MAIN 

5 CONTINUE  3 PCTVCY I J 1 -PCTVGl ( J I /FLOAT ( NIStJll  MAIN 

PCTVCYl J ) =( SQR  T ( PCTVCYIJ I) ) /(ME  ANSI J) ) MAIN 

6 WRITE  (6,150)  NISI J ) ,N2S( J I .MEANS! Jl , (PES( I , J) ,1 =1 .LMXIST1  MAIN 

00  7 J = 1 , NSAV  3 TOPCTV=PCTVCY( JI  + TOPCTV  3 N3 S=Nl S ( J) ♦ N3 S MAIN 

NAS  = N2S(  J l+NAS  MAIN 

7 CONTINUE  3 TOPCT V = 1 1 TOPC TV > / 1 Ft OA T I NSA V) ) ) *1 00.  MAIN 

IF  (NSAV.EQ.50)  WRITE  ( 6 , 1 5 1 » MAIN 

DO  9 I = 1 , LMX I ST  3 POOLVI  1 )*0.  3 NPLD(l)=0  MAIN 

DO  8 J = l , NS AV  3 IF  ( P E S ( I , J ) . E Q . 0. I GO  TO  8 MAIN 

NPLDl  I )=NPLD(  I )+N2S( J ) 3 P OOL V ( I ) =POOL V ( I ) ♦ F LO AT ( N2 S ( J) ) *PES  ( I , J ) MAIN 

3 CONTINUE  3 IF  (NPLD(l).EO.OI  GO  TO  9 MAIN 

POOLVI  I I -POOLVI  I ) /FLOAT! NPLDl I)  I MAIN 

9 CONTINUE  MAIN 

DO  11  I * 1 , LMX I ST  3 IF  ( POOL V(  I ) .LE .0.  ) GO  TO  10  MAIN 

POOLS! I l=SQRT( POOLVI I ) ) 3 GO  TO  11  MAIN 

10  POOLS!  I 1 = 0.0  MAIN 

11  CONTINUE  3 WRITE  (6,152)  t POOL V 1 1 ) , I =1 . LMXI ST)  MAIN 


1 

2 

3 

A 

5 

6 

7 

8 
9 

10 
11 
12 
13 
1 A 

15 

16 

17 

18 
19 
200 
210 
220 
230 
2A0 
250 
26  0 
270 
28W 

29  R 

30  R 

3 1 R 
32R 
33 

3 AR 

35 

36 

37 
38W 
39 
AO 
A1 
A2 
A3 

A AW 

A5 

A6 

A7 

A8W 

A9 

50 

51 

52 

53 
5A 

55 

56 
5T 
58W 
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WRITE  (6,154)  (POOLS) I), I •l.LMXISTI 
WRITE  (6,153)  (NPLD( I l,I*l,LMXIST) 

WRITE  (6,155)  TOPCTV.LMXIST.N4S 
00  15  IM.MAXIST 

DO  14  J = l , N SA V t IF  (PES( I, Jl.GE.O.O)  GO  TO  12  t GO  TO  13 

12  PEV(  I, J l*SQRT( PES(  I.JI  I t GO  TO  14 

13  P EV ( I, J 1-0.0 

14  CONTINUE 

15  CONTINUE  t WRITE  (6,90)  ( L I NSTR ( I ) , I =1 , LMX I ST  I 
DO  16  J - 1 , NS AV 

WRITE  (6,150)  NISI J ),N2S( J) .MEANS! J) , (PEV( I ,J) ,1=1 .LMXIST ) 

16  CONTINUE  t WRITE  (6,152)  ( POOL V < I ) , I *1 . LMX I ST  I 
WRITE  (6,154)  (POOLS( I), 1*1, LMXIST) 

WRITE  (6,153)  (NPLOI 1 ), I =1,LMXI ST) 

WRITE  (6,155)  TOPCTV, LMXIST, N4S 

17  NS  AV*0  S LMX  I ST  = 0 

18  IF  ( MAXRNO.EO.O)  STOP 
DO  19  I • 1 , MAX  I ST 

19  LINSTRI  I )*INSTR(  I ) $ IF  ( MAXI ST.GT. LMXI ST)  LMXI ST=MAX I ST 
NS  AV  *NS  AV ♦ 1 

DO  20  1*1,10 

20  P ES  ( I , NS  AV ) =0  • » RND*0. 

DO  22  I = 1 , M AXRND  t Y(  I )=Z< I )»U( I )»SMT(I 1=TMR(I)=RSTM(I)»0. 

J * INCRNDI  I ) l 1RNDELI I )=IDEL( J*l) 

DO  21  J* 1 , MAX  I ST 

21  CTL(  I,J)*RTL(  I,J)-0. 

22  IF  ( INCRNOt I ) .NE.O)  RND*RND+1.  $ IST*0. 

DO  23  1*1, MAX  1ST  t J = INC  I ST( I ) i I STDEL ( I ) *1  DEL ( J+l I 
IF  ( INCISTI  I ) .NE.O)  IST  = I ST* 1. 

SUMAI  I ) *COVA(  I )=VAR( I ) =ME ANA ( I ) =VAR ( I ) *COVNA ( I ) = PE ( I ) * PRB ( I ) *0 . 

S DEV ( I ) =0 - $ JPE(  I ) *0 
DO  23  J=1,MAXIST 

23  COVI  I, J )*0.  t TVAR *0 . 

DO  25  1*1, MAX  1ST  t IF  ( I NC I ST ( I ) . EQ. 0)  GO  TO  25 
DO  24  J* 1, MAXRND 

24  V(J)*A(J,  I)  S CALL  OUTLIR  ( V,  I NCRND  , MA  XRNO  , I ND  , K , VA ) $ VARd-VA 
SDEVI  I ) * SORT ( VAR(  I ) ) % PRB ( I ) *. 6745* SDE V ( I ) t TVAR=TVAR+VA 

IF  ( IND.NE.O)  CTLIK, I ) = • I • 

25  CONTINUE  * TVAR *T VAR / 1 ST  * TSDE V* SOR T ( T VAR ) t TPROBE* . 6745*TSDEV 
DO  27  1*1, MAXRND  $ IF  ( I NCRND ( I ) . EQ. 0)  GO  TO  27 

DO  26  J* 1 , MAX  1 ST 

26  V(J)*A(  I , J ) t CALL  OUTLIR  ( V, I NC I ST , MAXI  ST , I ND , K , VA ) 

IF  ( INO.NE.O)  RTL ( I,K)»'P* 

27  CONTINUE  t Nl=IST+.5  1 N2*RND+.5  t NM1*N2-1  t NM2*NM1-1  S TSUM=0. 
DO  29  I*1,MAXIST  S IF  ( I NC I ST ( I ) . E 0 . 0)  GO  TO  29 

DO  28  IRN0*1, MAXRND 

28  IF  ( INCRNDI  IRNDI.NE.O)  SUMA (II *SUMA ( 1 1 » A ( I RND , I) 

TSUM* TSUM +SUM A ( I)  t MEANAI I l*SUMA(l ) /AND 

29  CONTINUE  $ MEAN*TSUM/( IST*RNO)  » TC0V=0.  t KM*MAXIST-1 
DO  32  1*1, KM  t IF  ( INCISTI I I.EQ.O)  GO  TO  32  t K=I*1 

DO  31  J*K,MAX  1ST  t IF  ( I NC I ST ( J ) . EQ. 0)  GO  TO  31 
DO  30  IRND*1, MAXRND 

30  IF  I INCRNDI  IRNDI.NE.O)  C 0V( I , J I *C0 V ( I , J I ♦ A ( I RND , I ) * A ( I RND , J ) 

COVI J, I I = COVI  I, J ) »(RND*COV( 1 , J I- SUMAI I )* SUMAI J) ) I ( RNO* ( RNO-1 . 1 I 
COVAI  I )*COVA(  I l+COVI  I.J)  » COVAl  J)*COVA(  JUCOVII  .J) 

31  CONTINUE  t T COV *T COV+C OV A ( I ) 

32  CONTINUE  * IF  ( I NC I ST ( MA X I ST  I .NE . 0 ) TC OV*TCOV+C OVA  I MAXI  ST ) 
TCOV*TCOV/2. 

DO  36  1*1,MAXIST  t IF  ( I NC I ST ( I ) . E Q. 01  GO  TO  36 
COVNAI I )*TCOV-COVA( I ) t IF  (N1.EQ.2)  GO  TO  33 


MAIN  59W 
MAIN  60W 
MAIN  61W 
MAIN  62 
MAIN  63 
MAIN  64 
MAIN  65 
MAIN  66 
MAIN  67W 
MAIN  68 
MAIN  69W 
MAIN  70W 
MAIN  7 1W 
MAIN  72W 
MAIN  73W 
MAIN  74 
MAIN  75 
MAIN  76 
MAIN  77 
MAIN  78 
MAIN  79 
MAIN  80 
MAIN  81 
MAIN  82 
MAIN  83 
MAIN  84 
MAIN  85 
MAIN  86 
MAIN  87 
MAIN  88 
MAIN  89 
MAIN  90 
MAIN  91 
MAIN  92 
MAIN  93 
MAIN  94 
MAIN  95 
MAIN  96 
MAIN  97 
MAIN  98 
MAIN  99 
MA I N 100 
MAIN101 
MAIN102 
MA I N 103 
MA IN  104 
MAIN105 
MAIN  106 
MAIN107 
MAIN108 
MA  I N 109 
MAIN110 
MAIN1U 
MA IN l 12 
MA INI  13 
MAIN114 
M A I N 1 15 
MA INI 16 
MAIN117 
MAIN11B 
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PEI  I )«VAR(  I )-2.*C0VA(  l )/(  1ST-  1. I »2 . *C  0 VNA ( I ) / I ( I ST-1 . ) * 1 1 ST-2 . ) I MAIN119 
GO  TO  36  MAIN120 

33  PEI I)=VAR( II-TCOV  MA1N121 

DO  34  IT*1,MAXIST  3 IF  I INC  I ST  I I T ) . E Q. 01  GO  TO  34  3 IIS1-IIS2  MAIN122 

I I S2=  IT  MAIN123 

34  CONTINUE  MAIN124 

00  35  1T=1,MAXRND  MAIN125 

35  VI  IT  > = A(  IT,  I IS1 1 - A I IT,  I I S2I  MAIN126 

CALI  OUTUR  I V,  INCRND.MAXRND,  I ND , K A STR , VA I MAIN127 

IF  (INO.NE.O)  ASTR(KASTR)=**«  MAIN128 

J = 0 MAIN130 

36  PESI  I , NS  A V ) =P  E ( I)  3 NIS(NSAV)=N1  3 N2S(NSAV)=N2  3 ME ANS ( NS AV > = ME ANM A I N 1 29 
DO  37  I=1,MAXIST  3 IF  I ISC  1ST! I t.EQ.O)  GO  TO  37  * J=J  + l 
APE(  J ) = PEl  I 1 5 IPEI  J 1 = 1 

37  CONTINUE  » CALL  SORTXY  IAPE,IP£,J) 

DO  38  1 = 1, J V K = I PEI  I ) 

38  J P E ( K ) = I 3 PARAV  = 2.*TC0V/( IST*I IST-l.II 
00  45  1=1, M AXRND  l ACRINS=0.0  V ACR0SS(I)=0.0 
IF  I INCRNOI I ) .EO.O)  GO  TO  39  3 GO  TO  40 

39  ACROSS!  I ) =0.0  $ GO  TO  45 

40  00  42  JM.MAXIST  3 IF  I INC  I ST  I J ) . NE  . 0)  GO  TO  41  3 GO  TO  42 

41  ACROSS!  I)=ACROSS<  I MAI  I, J)  3 ACR I N S=AC R I NS+ 1 . 

42  CONTINUE  3 IF  I ACR IN S . EO . 0 . 0 ) GO  TO  43  t GO  TO  44 

43  ACROSSI  I )=0.0  $ GO  TO  45 

44  ACROSSI  I >=ACROSS<  I l/ACRINS 

45  CONTINUE  3 WRITE  I 6, 97  I I T I TL E I I ) , I = 1 . 1 6 I 
WRITE  (6,981  I INSTR I I I , I =1  ,MAXI ST) , A8CD 
WRITE  (6,99)  ( ISTNAMI  II,  1=1, MAXISTI 
WRITE  (6,100)  I ISTDELI  I I, I = 1,MAXIST) 

DO  46  1 = l.MAXRND 

46  WRITE  (6,101)  IRNOEL I I ) , ASTRI  I ) , I , I A I I , J) ,RTL I I , J) ,CTL( I , J) , J=1 , 

1 MAX  1ST ), ACROSSI  I ) 3 WRITE  (6,102)  I ME  ANA  1 1 I , I = 1 , MAX  I S T I 


WRITE  16,103)  (VARI  I ),  1 = 1, MAXI  ST) 

WRITt  (6,104)  ( SDEV(  l ) , I =1,MAXI ST) 

WRITE  (6,105)  (PRB(  I ),  I = 1,MAXIST) 

WRITE  (6,1061  MEAN, TVAR, TSOEV, TPROBE  t WRITE  (6,107) 
WRITE  16,1081  N1.N2  3 IF  I M A XRNO . LE . 1 5 ) GO  TO  47 
WRITE  (6,97)  TITLE 

47  WRITE  (6,109)  ( IMSTR ( I )• I =1,MAXI ST) 

WRITE  (6,110)  ( ISTNAMI I ), 1=1, MAXI  ST) 

WRITE  (6,111)  ( ISTOEL ( I ) , 1 = 1, MAXI STI 
DO  48  I=1,MAXIST 

48  WRIT,  (6,112)  I STOEL ( I ) , INSTR ( I ) , (COV( I , J)  , J = 1 ,MAXI ST) 
WRITE  (6,1)3)  ( COVAI I t , I =1,MAXI ST ) 

WRITE  (6,114)  ( COVNAI I ), I=l,MAXISTI 
WRITE  (6,1151  (PE ( I ), I =1 ,MAXI ST) 

DO  5!  1 = 1, MAX  1ST  3 IF  (PF(I)-O.)  50,50,49 

-9  PEP(  I )»SQ*T< Pt(  I ) ) 3 GO  TO  51 


DO  51  I-I.W'XIDT  3 IF  (PFIII-O.)  50,50,49 
-9  PEP(  I )»SQRT ( PE ( I ) ) 3 GO  TO  51 
50  P E P ( 11=0.0 

5!  CONI  INUE  » WRITE  (6,156)  ( PE P ( I I , I * 1 , M A XI  ST  I 3 IF  (PARAV)  52,52, 

52  PARS=0.0  * GO  TO  54 

53  PARS=SGRT< PAR AV ) 

54  WRITE  (6,116)  (JPC 11). 1=1, MAX  1ST)  i WRITE  (6,1171  TCOV, PARAV, PAR 
ASTR ( K ASTR I = ■ • \ L AST  = 0 3 !STPT  = 1 t ISTAT*0 

55  READ  (5,921  ( GRP ( 1 ) , I » 1 , 80 ) t ICHAR=0 

56  ICHAR  = ICHAP* 1 3 IF  ( I CHAR . EQ . 8 1 ) GO  TO  55  % J = GRP(ICHAR) 

IF  (J.EO.OI  GO  TO  56 

00  57  I*1.MAXIST  3 IF  ( .)  .FO.  I STNAMt  I I I GO  TO  58 

57  CONTINUE  t (=0  3 IF  (J.EO.'*')  GO  TO  59  3 IF  ( J. EQ. • , • ) GO  TO  65 
IF  ( J . EQ  - • . • ) GO  TO  64  3 1 S T A T = l 3 GO  TO  56 


TO  37  3 J=J+1  MAIN131 

MAIN132 
MAIN133 
MAIN134 
M A I N 1 35 

0.0  MAIN136 

MAIN137 

MAIN138 

TO  41  3 GO  TO  42  MAIN139 

♦1.  MAIN140 

0 TO  44  MAIN141 

MAIN  142 

MAIN143 

MAIN144W 

MAIN145W 

MA1N146W 

MAIN147W 

MA I N 148 

RTL( I , J) ,CTL( I, J) , J=1 , MAIN149W 

(I ) » I = l , MAXI S Tl  MAIN150W 

MAIN151W 
MA IN152W 
MAIN153W 

TE  (6,107)  MAIN154W 

TO  47  MAIN155W 

MAIN156W 
MAIN157W 
MAIN158W 
MAIN159W 
MAIN  160 

, J = 1 , MAXI  ST)  MAIN161W 

MAIN162W 
MAIN163W 
MAIN164W 
MAIN165 
MAI N 166 
M A I N 1 67 

Tl  3 IF  (PARAV)  52,52,53MAIN16SW 

MAIN  169 
MA I N 170 

(6,117)  TCOV, PARAV, PARS  MAIN171W 

0 MAIN172 


MAIN172 

MAIN173R 

MAIN174 

MA1N175 

MA I N 176 

MA1N177 

MA I N 1 78 
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58  IF  ( INCISTI  D.EQ.O)  ISTAT-1  MAIN179 

59  GO  TO  (60,61,62,63),  ISTPT  MAIN180 

60  11=1  $ ISTPT  = 2 % GO  TO  56  MAIN181 

61  12=1  S IF  ( 12. NE.O. AND. I2.EQ.il)  ISTAT«1  t ISTPT-3  t GO  TO  56  MAIN182 

62  13=1  * IF  ( 13. NE.O. AND. ( I3.EQ.il. OR. 13. EO. 12))  ISTAT  = 1 $ ISTPT=4  MAIN183 


GO  TO  56 

63  I S T AT  » l $ GO  TO  56 

64  L AST=  1 t IF  ( ISTPT. EQ.l)  GO  TO  1 


MA I N 184 
HAIN18S 
MAIN186 


65  IF  (ISTPT.NE.4)  ISTAT  = 1 t IF  (ISTAT.EQ.O)  GO  TO  67  t WRITE  ( 6 , 157 ) HA  I N187W 

66  IF  (LAST. NE.O)  GO  TO  1 t ISTPT=1  » ISTAT=0  * GO  TO  56  MAIN188 

67  IF  (13. NE.O)  GO  TO  68  l I T 1 = 1 ( IT2=MAXIST  » GO  TO  69  MAIN189 

68  IT  1= I 3 t IT 2= I 3 MAIN190 

69  00  89  T = IT1,  IT2  t IF  ( INC  I ST ( T ) . EQ. 0)  GO  TO  89  MAIN191 

00  70  I=1»MAXIST  MAIN192 

00  70  J=1,MAXIST  MA I N 193 

70  TTV(  I, J l=TTL(  I , J ) =TVL ( I , J ) =TLL ( I , J ) =R S V ( I , J ) =RSL ( I , J ) = S VL < I , J MAIN194 

1  l = SLL(  I, J )=0.  t FT  =0.  t IF  (12. NE.O)  GO  TO  71  * I SI  = 1 MAIN195 

IS2=MAXIST  t GO  TO  72  MAIN196 

71  IS  1= I 2 » IS2= 12  MA IN  197 

72  DO  83  KS  = IS1, IS2  * IF  ( I NC I ST ( S ) . EQ. 0)  GO  TO  83  MAIN198 

IF  (S.EQ.T)  GO  TO  83  S IF  (11. NE.O)  GO  TO  73  MAIN199 

IF  (I2.EQ.0)  GO  TO  74  * I R 1 = 1 S IR2=MAXIST  t GO  TO  75  MAIN200 

73  I R 1=  1 1 t IR2=  1 1 % GO  TO  75  MAIN201 

74  I R 1*5 ♦ 1 t IF  ( IR1.GT.MAXIST)  GO  TO  83  S IR2=MAXIST  MAIN202 

75  00  83  R*IR1,IR2  i IF  ( I NC I ST ( R ) . E Q. 0 ) GO  TO  83  MAIN203 

IF  ( R.EQ.S.OR.R.EO.T)  GO  TO  83  MAIN204 

DO  76  1 = 1, M AXRN 0 t IF  ( I NCRND ( I ) . E Q . 0)  GO  TO  76  MAIN205 

Y ( I)  = A(  I , R ) + A ( I,S)-MEANA(R)-MEANA(S)  i Z ( I ) =A ( I , R) -A ( I , S)  MAIN206 

U(  I )=A(  I,T)-( A(  I,R)  + A( I, S) )/2.  t SMT ( I ) =A ( I , S)-A ( 1 ,T)  MAIN207 

TMR(  I)  = A(  I , T ) — A ( I , R ) $ * S TM ( I ) = l A ( I , R ) + A ( I , S ) ♦ A ( 1 , T) 1 /3.  MAIN208 

76  CONTINUE  * CALL  OUTLIR  ( Y , I NCRNO , MA XRND , I NO , KY , V A ) MAIN209 

IF  (IND.NE.O)  YF ( KY ) = ' * ' $ CALL  OUTLIR  ( Z , I NCRNO , MAXRND , I ND , KZ , V A ) MA I N210 
IF  (IND.NE.O)  ZF(KZ)  = **t  i CALL  OUTLIR  ( U , I NCRNO , MAXR NO , I ND , KU , V A ) M A I N2 1 1 
IF  ( INO.NE.O)  UFIKU)****  MAIN212 

CALL  OUTLIR  ( SMT , INCRNO , MAXRND , I NO , K SMT , VA » MAIN213 

IF  (IND.NE.O)  SMTF(KSMT)«'*'  MAIN214 

CALL  OUTLIR  ( TMR , INCRNO, MAXRNO , I ND , K TMR , VA ) MAIN215 

IF  (INO.NE.O)  TMR F ( XTMR ) = • * • S S2 Y = VAR ( R ) ♦ VAR ( S I +2 . *C0 V ( R , S ) MAIN216 

S2Z  = VAR( Rl+VARI S I - 2. *COV ( R , S I MAIN217 

S2U  = VAR(  T)  + S2Y/4.-C0V(T,S)-C0V(T,R)  MAIN218 

S2SMT»VAR( S ) ♦ V AR ( T )-2.*C0V( S,T)  * S2 TMR= VAR ( T ) + V AR ( R ) -2 . *C0V ( T , R ) MAIN219 


SYZ*VAR( RI-VARI S I 1 SUV=C 0 V ( T , R ) -C 0 V ( T , S ) -S YZ/2 . 
S2RE=(S2Z-S2SMT  + S2TMR  )/2.  t S2SE  = ( S2Z+ S2  SMT- S2  TMRI /2  . 


MAIN220 

MAIN221 


S2TE*(-S2Z+S2SMT+S2TMR )/2.  i 52 X* ( CO V ( R , S ) +C OV ( S ,T ) 4C0V ( R , T ) ) / 3.  MAIN222 
RYZ=SYZ/SQRT( S2Y*S2Z ) t RU V=SUV/SQRT ( S2U* S2 Z ) MAIN223 

ZMEAN*MEANA(R )— ME ANA ( S ) S UME AN =ME ANA < T) - ( ME  ANA < R ) ♦ ME  ANA l S ) ) /2 . MAIN224 

SMTM*MEANA( S)-MEANA(T)  t TMRM =ME ANA ( T ) -ME  ANA ( R)  MAIN225 

TRSV=RYZ*SQRT(RN0-2. )/SQRT( l.-RYZ*RYZ)  MAIN226 

TRSL=ZMEAN*SQRT( RND)/S3RT( S2Z ) MAIN227 

TTSRV=(S2U/S2Z— .75)*SQRT(RND-2.)/SQRT(3.*(l.-RUV*RUV)*S2U/S2Z)  MAIN228 
TTSRL=UMEAN*SQRT( RNO) /SQRTI S2U)  * WRITE  (6,97)  TITLE  MAIN229W 

WRITE  (6,1181  INSTRIR  ), INSTRl S) ,INSTR( T) , I STNAM(R)  , I S TNAM ( S I , MAIN230W 

1 ISTNAM(T)  MAIN231W 

WRITE  (6,119)  ( IRNDELI I ) , I , A ( I , R ) , R TL ( I ,R)  , CTL ( I ,R) * A ( I ,S),RTL( I,  MAIN232W 

1 SI.CTLI  I , S ) , A ( I,TI,RTL(  I , T I , C TL (I , T) , RSTM ( I ) f Z ( I ) , ZF ( I ) , S MT ( I I , MAIN233W 

2 SMTF(  I ),TMR(  I ), TMRFI I ), I =1, MAXRND)  MAIN234W 

WRITE  (6,120)  ZMEAN, SMTM, TMRM  » WRITE  (6,121)  S2 Z , S2 SMT ,S2TMR  MAIN235W 

WRITE  (6,122)  S2RE.S2SE, S2TE  S WRITE  (6,123)  S2X  MAIN236W 

WRITE  (6,1241  N 1 I IF  (MAXRND-12)  78,78,77  MAIN237W 

77  WRITE  (6,971  TITLE  MAIN238W 
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78  WRITE  (6,125)  IN S TR  ( R ) , I N STR  ( S I , I N STR  ( T)  , I S TNAM  ( R > , I S TN AM  ( S ) , MAIN239W 

1 ISTNAM(T)  MAIN250W 

WRITE  (6,126)  ( IRNIDEl ( I ) , I, A( I ,R) ,RTL( I ,R) ,CTL( 1 ,R)  ,A( I ,S) ,RTL( I , MAIN251W 

1 S ) ,CTl(  I,S  ),  A(  I ,T)  ,RTL  ( I , T)  ,C  TL  ( I ,T)  , Y(  I ) , YF  ( I ) ,Z(  I ) , ZF  ( I ) ,U(  I ) , MAIN252W 

2 UF  ( lli I=1,MAXRND)  t ZERO=0.0  MAIN253W 

WRITE  (6,127)  MEANAI R ) ,MEANA( S) ,MEANA( T) .ZERO. ZMEAN.UME AN  KAIN255W 

WRITE  (6,128)  VAR(R ) , VAR  I S) , VAR ( T) ,S2Y,S2Z,S2U  MAIN255W 

WRITE  (6,129)  SYZ , SUV.RYZ ,RUV  MAIN256W 

WRITE  (6,130)  TRSV,TTSRV,T95(NM2) ,NM2,TRSl_,TTSRL,T95(NMl) ,NMl  MAIN257W 

TTV< S,R )=TTV< R, S )=TTSR V 1 T TL  ( S , 3 > =TTL ( R , S ) = TTSR L MAIN258 

RSV(R,S)=RSV( S,R)=TRSV  i R SL  < R , S > =R  SL  ( S . R ) = TR  SL  MAIN259 

IF  ( ABS( TRSV) .LT.T95INM2) ) GO  TO  79  $ WRITE  (6,131)  MAIN250W 

SVL( R,S) =SVL ( S,R ) ='*•  MAIN251 

79  IF  ( ABS( TRSL I .LT.T95INMI ) ) GO  TO  80  t WRITE  (6,132)  MAIN252W 

SLL(R,S)=SLLI S,R)=**'  MAIN253 

80  IF  ( ABS( TTSRV).LE.T95(NM2) ) GO  TO  82  MAIN255 

IF  l S2TE.LE.I S2RE  + S2SE  1/2. ) GO  TO  81  S T VL  ( R , S)  = T VL  ( S , R I = • -■  MAIN255 

WRITE  (6,135)  $ GO  TO  82  MAIN256W 

81  TVL(R,S)=TVL(S,R)='+'  » WRITE  (6,133)  MAIN257W 

62  IF  ( ABS( TTSRL ) .LE.T95INM1 ) I GO  TO  83  S WRITE  (6,135)  MAIN258W 

TLL( R,S  I =TLL<  S,R)='*'  MAIN259 

YF(KY )=ZF(KZ ) =UF( KU) =SMTF(KSMT) =TMRF (KTMR) =0.  t WRITE  (6,125)  N1  MAIN260W 

83  CONTINUE  t IF  (Il.NE.O)  GO  TO  66  t IF  (I2.NE.0)  GO  TO  66  MAIN261 

IF  (FT.NE.O.)  GO  TO  86  t FT=1.  i WRITE  (6,97)  TITLE  MAIN262W 

WRITE  (6,136)  S WRITE  (6,155)  T95(NM2),NM2  MAIN263W 

WRITE  (6,138)  ( INSTRI I), 1=1,MAXIST)  MAIN265W 

WRITE  (6,139)  ( ISTOELl I ) , I = 1 , MA X l ST ) MAIN265W 

DO  85  1 = 1, MAX  1ST  MAIN266 

85  WRITE  (6,150)  I ST  DEL ( I ) , IN S TR ( I ) , ( R S V ( I , J ) , S VL ( I , J I , J = 1 , MAX  I S T ) MAIN267W 

WRITE  (6,156)  i WRITE  (6,155)  T95(NM1),NM1  MAIN268W 

WRITE  (6,138)  < INSTRI I ), I =1,MAXIST)  MAIN269W 

WRITE  (6,139)  ( ISTDELI  I),  I=1,MAX1ST)  MAIN270W 

DO  85  I=1,MAXIST  MAIN271 

85  WRITE  (6,150)  I STDEL ( I ) , I N STR ( I ) , ( R SL ( I , J ) , SLL ( I , J ) , J= 1 , M AX  I ST ) MAIN272W 

WRITE  (6,157)  MAIN273W 

86  WRITE  (6,97)  TITLE  t WRITE  (6,136)  MAIN275W 

WRITE  (6,137)  INSTRI T) ,T95(NM2) ,NM2  MAIN275W 

WRITE  (6,138)  ( INSTRI I ), 1=1, MAXIST)  MAIN276W 

WRITE  (6,139)  ( ISTDELI  I I, l = l,MAXIST|  MAIN277W 

DO  87  I=1,MAXIST  MAIN278 

87  WRITE  (6,1501  I STDEL ( I I , I NSTR ( I ) , ( TT VII , J ) , T VL ( I , J ) , J= 1 , MAX  I ST ) MAIN279W 

WRITE  (6,151)  i WRITE  (6,152)  T95 ( NM1 ) , NM1 , ( I NSTR 1 1 I , I = 1 , M AX  I S T ) MAIN280W 
WRITE  (6,139)  ( ISTDELI I ), 1*1, MAX1ST)  MAIN281W 

DO  88  1 = 1, MAX  1ST  MAIN282 

88  WRITE  (6,150)  I ST  DEL ( I I , INSTR ( I ) , ( TTL ( I , J ) , T LL ( 1 , J ) , J= 1 , MAX  I ST  I MAIN283W 

WR ITE  ( 6, 153)  MAIN285W 

89  CONTINUE  t GO  TO  66  MAIN285 

MA1N286 


90 

FORMAT 

( 1HI,  12X, 59HSUMMAR Y OF  STANDARD  DEVIATION  OF 

PRECISION  ERR0MAIN287 

1RS//1 6X 

, 5H IN  ST,  5X , 6HP  01  NT  S, 6X , 5HME  AN , 1 0 ( 2 X , A8 ) ) ) 

MAIN288 

91 

FORMAT 

(8011) 

MA1N289 

92 

FORMAT 

( 80 A 1 ) 

MAIN290 

93 

FORMAT 

( 8A10) 

MAIN291 

95 

FORMAT 

( 1 0A8 ) 

MAIN292 

95 

format 

( 10F8.1I 

MAIN293 

96 

FORMAT 

(2 12) 

MAIN295 

97 

FORMAT 

( 1H1, 15X,8A10,/,IH  , 15X.8A10) 

MAIN295 

98 

FORMAT 

( 1H0,  7X, 5HDATA,  1 0 ( 2 X , A 8 , 1 X ) , 2 X , A 8 ) 

MAIN296 

99 

FORMAT 

( 6X, 5HP0INT, 10( 7X.Al.3X) ) 

MA1N297 

100 

FORMAT 

( 1 IX, 1 0 ( 2X , A8, 1X1 ) 

MAIN298 

101  FORMAT  t A8, Al, 12, 10(F9.2,2A1) ,F9.1) 

102  FORMAT  t 1 HO, 7X , 4HME AN , 10< F 9. 2 , 2X )) 

103  FORMAT  ( 3X,8HVARIANCE« 10(F9.3,2X)) 

104  FORMAT  (3X.8HSTND  DE V , l 0( F 9. 3 , 2X ) 1 

105  FORMAT  ( 3X, 8HPR0B  ERR , 10( F9. 3, 2X )) 

106  FORMAT  ( 15H0  GRAND  ME AN  = , F 1 0. 3 , 2 OH 


MAIN299 

MA1N300 

MAIN301 

MA1N302 

MAIN303 

AVERAGE  VARIANCE®, F10.4,12HMAIN304 


1 STND  OEV®, F7.4, 12H  PROB  ERR=,F7.41  MAIN305 

107  FORMAT  ( 84H0MEASUREMENTS  FOUND  TO  BE  OUTLIERS  AT  A 95  CONFIDENCE  MAIN306 
ILEVEL  ARE  FOLLOWED  BY  I OR  P./130H  I INDICATES  THE  MEASUREMENT  WASMAIN307 

2 FOUND  TO  BE  AN  OUTLIER  WHEN  COMPARED  TO  MEASUREMENTS  OF  OTHER  0ATMAIN308 
3 A POINTS  BY  THE  SAME  I N S TRUMENT . /7 3H  P INDICATES  THE  MEASUREMENT  WMAIN309 
4AS  FOUND  TO  BE  AN  OUTLIER  WHEN  COMPARED  TO  , 57H ME ASURE MENTS  DF  THEMAIN310 

5 SAME  DATA  POINT  BY  OTHER  I N STR UME NT S. /1 03H0*  INDICATES  AN  OUT L I ERMA  I N3 1 1 

6 WHEN  CONSIDERING  THE  DIFFERENCE  BETWEEN  INSTRUMENTS.  (TWO  I NSTRUMM A I N3 12 

7 ENT  CASE  ONLY  ) ) MAIN313 

108  FORMAT  ( 1H0,  I2.18H  INSTRUMENTS  WITH  ,I2,45H  DATA  POINTS  EACH  WERE  MAIN314 

1USED  IN  THIS  ANALYSIS.!  MAIN315 

109  FORMAT  ! 1H0, 40X, 17HC0VAR IANCE  MA TR I X/ , 1H0 , 1 6X , 1 0 (2 X , A8 1 ) MAIN316 

110  FORMAT  ( 14X, IOC 9X , A 1 ) ) MAIN317 

111  FORMAT  ( 16X, 10( 2X, A8) ) MAIN318 

112  FORMAT  ( 2A8,  10(  1X,F9.4|)  MAIN319 

113  FORMAT  ( 17H0  COV  INCLO  I N ST , 1 0 ( 1 X ,F 9. 4!)  MAIN320 

114  FORMAT  ( 17H0  COV  EXCLD  INST, 10( 1X.F9.4) I MAIN321 

115  FORMAT  I 1 2X , 1H2 / 1 7H  EST  (SIGMA  E 1 ) . 1 0 1 1 X ,F 9. 4 ) ) MAIN322 

116  FORMAT  ( 17H0  PRECISION  R ANK , 10 ( I 8 , 2 X!)  MAIN323 

117  FORMAT  ( 2 2H0  TOTAL  COVAR I ANCE = , F l 0. 4 , 4X , 2 OHP ARAMETER  VARIANCE  *MAIN324 

1F10.3,2X, 16HPARAM  STD  DEV.  »,F8.4)  MAIN325 

118  FORMAT  (12H  D A TA , 3 ( 1 X , A 8 , 2 X ) ,4X ,4H ME  A N , 1 0 X , 3HR-S ,8 X , 3HS-T , 8M A I N326 

IX, 3HT-R, /6X, 5HP0INT, 5X, 2HR ( , A 1 , 1H ) , 7X  »2HS ( , A 1 , 1HI7X.2HT ( , Al , 1H1/ ) MAIN327 


MAIN328 
MAIN329 
MAIN330 
R =,F11.4MAIN331 
MAIN332 


119  FORMAT  I ( A8, IX, 12,31 F9.1.2A1) ,F9.2,2X,3(F10.l,Al) ) ) 

120  FORMAT  ( 1H0, 51X,4HMEAN,3Fll.3) 

121  FORMAT  ( 47X,  8HVAR  IANCE-,  3F  11.3) 

122  FORMAT  I 1H0,28X, 1H2/37H  EST  INST  VARIANCES  SIGMA  (El  ) 

1 , 6H  S =,F11.4,6H  T =,F11.4) 

123  FORMAT  ( 1H0, 37X , 1H2/41H0E ST  PARAMETER  VARIANCE  SIGMA  (E  SUB  X)  =,FMAIN333 

110.4)  MAIN334 

124  FORMAT  ( 55H  I,  P,  ANO  * FOLLOW  OUTLIERS  AT  A .95  CONFIDENCE  LEVEL. MAIN335 

1/59H  I ANO  P INDICATE  MEASUREMENTS  FOUND  TO  8E  OUTLIERS  IN  THE  .I2MAIN336 
2, 2 1H  INSTRUMENT  ANALYSIS. ,/77H  * FOLLOWS  A VALUE  FOUND  TO  BE  AN  0UMAIN337 
3TLIER  WHEN  COMPARED  TO  OTHER  DATA  POINTS.)  MAIN338 

125  FORMAT  ( l HO, 7X , 4HDAT A , 3 ( 1 X , A 8 , 2 X ) , 9 X , 1 H Y , 7X ,4HZ= VI , 9X , 2 HU1 , / , 6X , 5HMA I N339 

1P0INT.5X, 2HR( , Al, 1H) , 7X,2HS( , A 1 , 1 H ) , 7X , 2HT ( , Al , 1 H ) ) MAIN340 

126  FORMAT  ((  A8,  I3,3(F9.1,2A1),3(F10.1,AIU)  MAIN341 

127  FORMAT  ( IH0,7X,4HMEAN,3(F9.3,2X),3(F10.3,1X))  MAIN342 

128  FORMAT  ( 3X , 8HV AR I ANCE , 3( F 9. 4, 2X ) , 3 1 F 1 0. 4 , 1 X ) ) MAIN343 

129  FORMAT  ( 10H0  S ( YZ ) * , F 8 . 4, 9H  S ( U V ) = , F 8. 4 , 9H  R < YZ ) = ,F8 . 4 , 9H  MAIN344 

IR(UV)*,F8.4)  MAIN345 

130  FORMAT  (17H0  TISTNDS  V AR ) = , F 8 . 3 , 1 5H  TITEST  V AR ) = , F8 .3 , 8H  T.9MAIN346 

15»,F8.3,2H  ( , I 2, 6H  D.F.I/17H  TISTNDS  B I A S ) = , F8 . 3 , 15H  TITEST  BIAMAIN347 
2S)=,F8.3,8H  T.95*,F8.3,2H  ( , 1 2 , 6H  O.F.)/)  MAIN348 

131  FORMAT  I 70H  THE  VARIANCES  OF  THE  STANDARD  INSTRUMENTS  ARE  S I GN I F I CM A IN349 

1ANTLY  DIFFERENT)  MAIN350 

132  FORMAT  ( 66H  THE  MEANS  OF  THE  STANDARD  INSTRUMENTS  ARE  S IGN I F I C ANTLM A I N351 

1Y  DIFFERENT)  MAIN352 

133  FORMAT  ( 69H  THE  TEST  INSTRUMENT  IS  SIGNIFICANTLY  MORE  PRECISE  THANMAI N353 

1 THE  STANDARDS)  MAIN354 

134  FORMAT  I 69H  THE  TEST  INSTRUMENT  IS  SIGNIFICANTLY  LESS  PRECISE  THANMAIN355 

1 THE  STANOAROS)  MAIN356 

135  FORMAT  1 69H  THE  MEAN  OF  THE  TEST  INSTRUMENT  DIFFERS  SIGNIFICANTLY  MAIN357 

1FR0E  STANDARDS)  MAIN358 


136  FORMAT  ( 1 6H0  T~ VALUE  SUMMARY)  MAIN359 

137  FORMAT  ( 7 9H0T  - VALUES  FOR  TEST  INSTRUMENT  HAVING  VARIANCE  S I GN  I F I C AM  A I N 360 

1NTLY  FROM  THE  ST ANDAR OS . / 1 7H  TEST  INSTRUMENT  , A8 , 1 0 X , 5H T . 95 = , F8 . 3 , M A I N36 1 
22H  ( , 12. 6H  D.F.  ) ) MAIN362 

138  FORMAT  ( 1H0.  10X, 5HSTNDS.  I 0( 2 X , A 8 ) ) MAIN363 

139  FORMAT  ( 16X,  10(  2X.A8M  MAIN364 

140  FORMAT  ( 2A8, 101 F9.3, A1  ))  MAIN365 

141  FORMAT  ( 68H0+  INDICATES  THE  TEST  INSTRUMENT  IS  MORE  PRECISE  THAN  TMAIN366 

1  HE  STANDARDS. /68H  - INDICATES  THE  TEST  INSTRUMENT  IS  LESS  PRECISE  MAIN367 
2T HAN  THE  STANDARDS.)  MAIN368 

142  FORMAT  ( /////54H0T- VALUES  FOR  TEST  INSTRUMENT  HAVING  SIGNIFICANT  8MAIN369 

1IAS../6H  T.95  = , F8.3, 2H  (.I2.6H  D. F . ) , / 1 HO , 7 X , 8H  STNDS.10I2X.A8)  1MAIN370 

143  FORMAT  (85H0*  INDICATES  THERE  IS  SIGNIFICANT  BIAS  BETWEEN  THE  TESTMAIN371 

1 INSTRUMENT  AND  THE  STANDARDS.)  MAIN372 

144  FORMAT  ( 65H0T- V 4L UE S FOR  STANDARDS  HAVING  SIGNIFICANTLY  DIFFERENT  MAIN373 

1VARIANCES./6H  T . 9 5 = , F 8 . 3, 2H  I , I 2 . 6H  D.F.))  MAIN374 

145  FORMAT  (///// 48H0T-VAL UE  FOR  SIGNIFICANT  BIAS  BETWEEN  S T AND ARDS . / 6M A I N 375 

1H  T . 95- » F8.3. 2H  (,I2,6H  D.F.))  MAIN376 

146  FORMAT  I87H0*  INDICATES  THERE  IS  A SIGNIFICANT  DIFFERENCE  IN  THE  VMAIN377 


1ARIANCES  OF  THESE  TWO  STANDARDS.) 
147  format  (69H0*  INDICATES  THERE  IS  A 
1TW0  STANDARDS. ) 


MAIN378 

SIGNIFICANT  BIAS  BETWEEN  THESE  MAIN379 

MAIN380 


148  FORMAT  (61H0THE  THEORY  FOR  THE  THREE  INSTRUMENT  ANALYSIS  IS  PR ES ENM A I N 38 1 

1TED  IN/34H0  ARDC  TECHNICAL  REPORT  NO.  11/57H  THE  ST  AT  I S T I M A I N 382 

2CAL  COMPARISON  OF  MEASURING  I N S TR UME N TS/ 30H  F.  E.  GRUBBS  AND  MAIN383 

3J . F.  05HBRY0N/15H  JULY  1971)  MAIN384 

149  FORMAT  (1H1.12X.IH2/25H  E ST  (SIGMA  El)  SUMMAR Y / / ( 6 X , 4H I NST , 4X , 6HM A I N 385 

IPO  I NTS, 6X , 4HMEAN,  10 ( 2X, A 8 ) ) ) MAIN386 

150  FORMAT  ( 2110, 11F10.5I  MAIN387 

151  FORMAT  ( 59H  THERE  IS  NOT  ENOUGH  STORAGE  TO  SUMMARIZE  MORE  THAN  50  MAIN388 

1RUNSI  MAIN389 

152  FORMAT  (30H0P00LED  VARIANCE  IN  PRECIS  E RR  , 1 X , 1 OF  1 0 . 5 ) 

153  FORMAT  ( 24H  NUMBER  OF  POINTS  P OOL E D 5X , 1 01 1 0 ) 

154  FORMAT  (30H0PO0LED  STND  DEV  IN  PRECIS  E RR . 1 X , 1 OF 1 0. 5 1 

155  FORMAT  ( 88H0ST ANDARD  DEVIATION  OF  PRECISION  ERRORS  AS 


MAIN390 
MAIN391 
M A I N 392 
PERCENT  0FMAIN393 


1 LEVEL  OF  CHARACTERISTIC  (P00LED1/11H  PERCENT  =,F8.5,9H  EASED  0N.MAIN394 


215, 15H  INSTRUMENTS  0N.I5.11H  OCCASIONS.) 

156  FORMAT  ( 17H  EST  (SIGMA  E1),10F10.5) 

157  FORMAT  ( 50H1  IMPROPERLY  DESCRIBED  GROUP  FOR  3 INSTRUMENT  CASE.) 
END 

SUBROUTINE  OUTLIR  ( V , I , N , I ND , K , VAR ) 

DIMENSION  V(N), I(N),R95( 50) 

DATA  R95/0..0., 

0494. . 1270. .2032.. 2696.. 3261 . . 3 742 . . 4 1 54 . . 45 1 1 , 

5097. . 5340. .5559.. 5755.. 593 3.. 6 095. .6243.. 6379, 

.6504, .6 621, .6728, .6829, .692 3. .7008, .7 08 7, .7161, 

.7232, .7300, .7365, . 742 7 , . 7486 , . 7 54 3 , . 7 597 , .7649 , 

7744. .  7788. . 7829. . 7869.. 7908. . 7946 .. 7982 , 


MAI N395 
MAIN396 
MAIN397 
MAIN398- 


.0027, , 
.4822, 


. 7698, . 


C 
C 
C 
C 

c 

C .8017, .8050, .8082, .8113, 

AI=SUM=VDM=SS=SUMl=SSl=INDsOS  DO  1 
A I = A I + l • S SUM  = SUM ♦ V ( J ) 

1 CONTINUE*  VB=SUM/AIt  DO  3 J = 1,N* 


8143, .8172/ 

J*1 , NS  IF  (I ( JI.EO.O)  GO  TO 

IF  (I (JI.EO.O)  GOTO  3 
VD  = ABS  ( V(  J l-VB  ) S IF  (VO.LT.VDM)  GO  TO  2S  K=*JS  VDM  = VD 

2 SS  = SS*V( J )*V( J) 

3 CONTINUES  SUM  l = SUM- V ( K ) S SS 1 = S$- V< K I * V ( K ) S SS=5S-  SUM*SUM/AI 

SS1  = SS1-  SUM1*SUM 1 /( A I-l . ) S VAR  *SS/  ( A I -1 . } 

IF  (SS.EO.O.)  RETURNS  R = SS1/SSS  L=AI*.5S  1 F ( L . LT . 3 ) RE  TURN 
IF  ( R.GT .R95< L ) ) RETURNS  IND=1*  RETURNS  END 


1 


**** 

OUTLIR 
OUTLIR 
OUTLIR 
OUTLIR 
OUTLIR 
OUTLIR 
OUTLIR  8 
OUTLIR  9 
OUTL I R 1 0 
OUT  L I R 1 1 
OUT  L I R1 2 
OUTL  IR13 
OUTL  IR14 
OUTL  IR1 5 
OUTL IR16 
OUTL  IR1T 
0UTLIR18 


SUBROUTINE  SCR T X Y I X , V , N I SORTXV  l 

C I MEN  S I ON  X(NltYIN)  SORTXV  2 

H.N  SORTXY  3 

1 m.m/2  SORTXY  A 

IFIM.EQ.OREIURN  SORTXY  5 

K»N-M*l  SCR T X Y 6 

j.|  SORTXY  7 

2 |,j  SORTXY  * 

3 l »I»M  SORTXY  S 

IFI  X ( I I .O.T.XIL  ) (GCTO  5 SORTXYIO 

4 j.j*l  SORTXY 1 1 

IF(J-K)2,l,l  SCRTXY12 

5 T.XIH  SORTXY 1 3 

X ( L I -XI ( I SORTXY 14 

XI  I ) * T SCRTXY1S 

T.YILI  SORTXYlt 

Y I L I aYI I ) SORT XY 1 7 

Y ( I ) a T SORTXY18 

INTERCHANGE  THE  I-TH  AND  L-TH  ELEMENTS  OF  ADDITIONAL  VECTORS.  S0RTXY19 

6 I . i-n  S0RTXY2C 

IFI I ) At  A » 3 SORT  XY  2 1 

ENO  S0RTXY22 
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Appendix  B 


Sample  Data  Input 

* DATA 

12  9 

lllllllll 

liiiiomin 

VELOCIMETER  OATA  , WAALSDORP  PROVING  GROUND  SERIES  13 
ESTIMATES  AND  RANKING  OF  PRECISION  OF  MEASUREMENTS  INSTRUMENTS! 


COUNTER 

FBI01 

COMP 

FBI  02 

FOTOCEL 

TERMA1 

TERMA2 

NM87  A 

NM878 

733.0 

732.0 

731.4 

733.3 

732.5 

715.50 

730.8 

732.4 

733.3 

729.6 

728.6 

737.2 

729.5 

729.0 

724.30 

728.7 

725.2 

729.7 

731.2 

730.2 

729.7 

731.3 

730.6 

727.50 

730.  4 

730.8 

731  .5 

734.6 

733.9 

733.2 

734.7 

734.0 

728.80 

734.0 

000.0 

734.8 

735.1 

734.  1 

733.4 

735.0 

734.5 

730.20 

734.9 

733.4 

735.1 

731.6 

0.0 

730.0 

731.3 

730.9 

723.60 

730.  5 

730.3 

731.6 

727.9 

727.1 

726.4 

727.8 

727.3 

722.20 

727.0 

726.9 

728.1 

731.7 

730.7 

730.0 

731.7 

731.1 

726.50 

730.5 

730.5 

732.2 

729.1 

728.4 

727.8 

729.  1 

728.  5 

723.80 

729.0 

000.0 

729.5 

728.1 

729.1 

727.0 

728.2 

727.6 

726.6 

721.80 

000.0 

728.4 

728.8 

728.0 

727.4 

729.0 

728.2 

712. 70 

728.2 

727.8 

729.3 

730.5 

729.7 

729.2 

730.6 

729.9 

72  8.4 

729.0 

730.6 

731.2 

12  9 

111110101 

nnioiniii 

VELOCIMETER  DATA  , WAALSDORP  PROVING  GROUND  SERIES  13 


DATA  GATHERED 

FROM  MULT INAT10NAL 

INSTRUMENTATION 

( WI  TH 

TWO  INS. 

DELETE01 

counter 

FBI01 

COMP 

FBI  02 

FOTOCEL 

TERMA1 

TERMA2 

NM87  A 

NM8  7B 

733.0 

732.0 

731.4 

733.3 

732.5 

715.50 

730.8 

732.4 

733.3 

729.6 

728.6 

737.2 

729.5 

729.0 

724.30 

728.7 

725.2 

729.7 

731.2 

730.2 

729.7 

731.3 

730.6 

727.50 

730.4 

730.8 

731.5 

734.6 

733.9 

733.2 

734.7 

734.0 

728.80 

734.0 

000.0 

734.8 

735.1 

734.1 

733.4 

735.0 

734.5 

730.20 

734.9 

733.4 

735.1 

731.6 

0.0 

730.0 

731.3 

730.9 

723.60 

730.5 

730.3 

731.6 

727.9 

727.1 

726.4 

727.8 

72  7.3 

722.20 

727.0 

726.9 

728.1 

731.7 

730.7 

730.0 

731.7 

731.  1 

726.50 

730.5 

730.5 

732.2 

729.1 

728.4 

727.8 

729.  . 

728.5 

723.80 

729.0 

000.0 

729.5 

728.1 

729.  1 

727.0 

728.2 

727.6 

726.6 

721.80 

000.0 

728.4 

728.8 

728.0 

727.4 

729.0 

728.2 

712.70 

728.2 

727.8 

729.3 

730.5 

729.7 

729.2 

730.6 

729.9 

728.4 

729.0 

730.6 

731.2 

12  9 

111000000 

union  ii  u 

VELOCIMETER  DATA  , WAALSDORP  PROVING  GROUND  SERIES  13 

3 INSTRUMENT  C AS E ( COMP AR ING  2 INSTRUMENTS  OF  KNOWN  PERFORMANCE  AGAINST  UNKNOWN) 


COUNT  ER 

FBI01 

COMP 

F8  102 

FOTOCEL 

TER  MAI 

TERMA2 

NM87A 

NM87B 

733.0 

732.0 

731.4 

733.3 

732.5 

715.50 

730.8 

732.4 

733.3 

729.6 

728.6 

737.2 

729.5 

729.0 

724.30 

728.7 

725.2 

729.7 

731.2 

730.2 

729.7 

731.3 

730.6 

72  7.50 

730.4 

730.8 

731.5 

734.6 

733.9 

733.2 

734.7 

734.0 

723.80 

734.0 

000.0 

734.8 

735.1 

734.  1 

733.4 

735.0 

734.5 

730.20 

734.9 

733.4 

735.1 

731.6 

0.0 

730.0 

731.3 

730.9 

723.60 

730.  5 

730.  3 

731.6 

727.9 

727.1 

726.4 

727.8 

727.3 

722.20 

727.0 

726.9 

728.1 

731.7 

730.7 

730.0 

731.7 

731.1 

726.50 

730.5 

730.5 

732.2 

729.1 

728.4 

727.8 

729.1 

728.5 

723.80 

729.0 

000.0 

729.5 

728.1 

729.1 

727.0 

728.2 

727.6 

726.6 

721.80 

000.0 

728.4 

72e.8 

728.0 

727.4 

729.0 

728.2 

712.70 

728.2 

727.8 

729.3 

730.5 

729.7 

729.2 

730.6 

729.9 

728.4 

729.0 

730.6 

731.2 

ABC. 


distribution  list 


Organization 


No.  of 
Copies 


Organization 


.2  Commander 

Defense  Documentation  Center 
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Redstone  Arsenal,  AL  35809 

1 Commander 

US  Army  Tank  Automotive 
Development  Command 
ATTN:  DRDTA-RWL 

Warren,  MI  48090 
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US  Army  TRADOC  Systems 
Analysis  Activity 
ATTN:  ATAA-SA 
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NM  88002 

1 Commandant 

US  Army  Logistics  Center 
Fort  Lee,  VA  23801 

1 Commander 

US  Army  Combat  Development 
Experimentation  Command 
Fort  Ord,  CA  93941 

1 Commandant 

US  Army  Artillery  5 Missile  Sch 
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4 Commander 
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